Stochastic approximations for finite-state Markov chains

Finite-State Markov Chains for Multiple Sequences

We consider the analysis of sets of categorical sequences consisting of piecewise homogeneous Markov segments. The sequences are assumed to be governed by a common underlying process with segments occurring in the same order for each sequence. Segments are defined by a set of unobserved changepoints where the positions and number of changepoints can vary from sequence to sequence. We propose a ...

Series Expansions for Finite-state Markov Chains

This paper provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

Stochastic Resonance in Two-State Markov Chains

The notion of Stochastic Resonance appeared about twenty years ago in the works of Benzi et al. [1] and Nicolis [2] in the context of an attempt to explain the phenomenon of ice ages. The modern methods of acquiring and interpreting climate records indicate at least seven major climate changes in the last 700,000 years. These changes occurred with the periodicity of about 100, 000 years and are...

Stochastic Resonance in Two-State Markov Chains

Finite-state Markov Chains Obey Benford's Law

A sequence ofreal numbers (xn) is Benford if the significands, i.e., the fraction parts in the floating-point representation of (x ), are distributed logarithmically. Similarly, a discrete-time irreducible and aperiodic fi­ n nite-state Markov chain with transition probability matrix P and limiting matrix P' is Benford if every com­ ponent of both sequences of matrices (pn P') and (pn+1 pn) is ...